Papers with task-specific representations
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings were static, requiring all senses of a polysemous word to share the same representation. |
| Approach: | They found that the contextualized representations of all words are not isotropic in any layer of the contextualizing model. |
| Outcome: | The results show that the representations of all words are not isotropic in any layer of the contextualizing model. |
To See a World in a Spark of Neuron: Disentangling Multi-Task Interference for Training-Free Model Merging (2025.emnlp-main)
Copied to clipboard
Zitao Fang, Guodong Du, Shuyang Yu, Yifei Guo, Yiwei Zhang, Yiyao Cao, Jing Li, Ho-Kin Tang, Sim Kuan Goh
| Challenge: | Existing approaches to model merging ignore the fundamental roles of neurons, connectivity and activation. |
| Approach: | They propose a framework that relies on neuronal mechanisms to mitigate task interference . they decomposed task-specific representations into two complementary subspaces . their results offer new insights into mitigating task interference and improving knowledge fusion . |
| Outcome: | The proposed framework reduces task interference within neurons and improves knowledge fusion. |
Task-Specific Information Decomposition for End-to-End Dense Video Captioning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for label assignment fail to ensure relevance of localization information to descriptions. |
| Approach: | They propose a decomposed dense video captioning framework that derives localization and captioning queries from event queries, enabling task-specific representations while maintaining inter-task collaboration. |
| Outcome: | Experiments on YouCook2 and ActivityNet Captions show that the proposed framework achieves state-of-the-art performance. |
DATA-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve sentence representation learning (SRL) ignore the potential interference problems across tasks and instances. |
| Approach: | They propose a multi-task instruction tuning method that arranges the order of multi- task data for training to minimize interference risks. |
| Outcome: | The proposed method can boost the performance of state-of-the-art methods. |
Impartial Multi-task Representation Learning via Variance-invariant Probabilistic Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on balancing loss or gradients but fail to address this issue due to the representation discrepancy in latent space. |
| Approach: | They propose a framework that harmonizes representation spaces across tasks to ensure impartial learning by harmonizing representation spaces. |
| Outcome: | The proposed framework outperforms 12 representative methods under the same multi-task settings, especially in heterogeneous task combinations and data-constrained scenarios. |